« -- 

48721 vl 23/12/03 



U 015110-2 



MULTIFUNCTIONAL INTEGRATED IMAGE SENSOR AND APPLICATION TO 
VIRTUAL INTERFACE TECHNOLOGY 

5 

REFERENCE TO CO-PENDING APPLICATIONS 
This application claims priority from the following co-pending U.S. 
Patent Applications: 

U.S. Provisional application serial number 60/392,376, entitled "Virtual 
10 Interface Implementation and Application", filed June 26, 2002 and U.S. Provisional 
application serial number: 60/438,327, entitled "Multifunctional Integrated Image 
Sensor and Application to Virtual Interface Technology", filed January 7, 2003. 



15 BACKGROUND OF THE INVENTION 

The following patents and publications are believed to represent Hie 
current state of the art: 

U.S. Patents: 6,377,238; 6,281,238; 5,767,842; 5,457,550; 5,181,181 and 

4,843,568. 

20 Foreign Patent Documents: WO 00/21024; EP 0982 676 Al; DE 298 02 

435 Ul; WO 01/93182 and WO 02/054169. 
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SUMMARY OF THE INVENTION 
The present invention relates to data input methods and apparatus 

generally. 

There is thus provided in accordance with a preferred embodiment of the 
5 present invention an electronic camera including an electronic imaging sensor providing 
an output representing an imaged field, a first imaging functionality employing the 
electronic imaging sensor for taking a picture of a scene in the imaged field, a second 
imaging functionality employing the electronic imaging sensor for data entry responsive 
to user hand activity and a user-operated imaging functionality selection switch 
10 operative to enable a user to select operation in one of the first and second imaging 
functionalities. 

There is also provided in accordance with another preferred embodiment 
of the present invention a portable telephone including telephone functionality, an 
electronic imaging sensor providing an output representing an imaged field, a first 

15 imaging functionality employing the electronic imaging sensor for taking a picture of a 
scene in the imaged field, a second imaging functionality employing the electronic 
imaging sensor for data entry responsive to user hand activity and a user-operated 
imaging functionality selection switch operative to enable a user to select operation in 
one of the first and second imaging functionalities. 

20 There is further provided in accordance with yet another preferred 

embodiment of the present invention a digital personal assistant including at least one 
personal digital assistant functionality, an electronic imaging sensor providing an output 
representing an imaged field, a first imaging functionality employing the electronic 
imaging sensor for taking a picture of a scene in the imaged field, a second imaging 

25 functionality employing the electronic imaging sensor for data entry responsive to user 
hand activity and a user-operated imaging fimctionality selection switch operative to 
enable a user to select operation in one of the first and second imaging functionalities. 

In accordance with another preferred embodiment of the present 
invention the electronic imaging sensor includes a CMOS sensor. 

30 Preferably, the first imaging functionality incorporates a first lens 

arrangement upstream of the electronic imaging sensor. Additionally, the second 
imaging functionality incorporates a second lens arrangement upstream of the electronic 
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imaging sensor. 

In accordance with yet another preferred embodiment of the present 
invention the second imaging functionaHty incorporates an infrared Hght generator for 
defining a light beam, impingement of which resulting from the user hand activity 
5 produces a light pattern representative of the data. Additionally, the second imaging 
ftmctionality also incorporates at least one infrared passing light filter associated with 
the electronic imaging sensor. 

In accordance with still another preferred embodiment of the present 
invention the second imaging ftmctionality incorporates a processor receiving an output 
10 from the electronic imaging sensor for providing a data entry output. Additionally, the 
processor is incorporated together with the electronic imaging sensor on a single chip. 

In accordance with another preferred embodiment of the present 
invention the first fimctionality is a color photography ftmctionality. Additionally or 
alternatively, the second fimctionality is an infixed beam impingement sensing data 
1 5 entry ftmctionality. 

In accordance with yet another preferred embodiment of the present the 
user-operated imaging ftmctionality selection switch is a mechanically responsive 
switch which selectably mechanically associates the first and second lens arrangements 
upstream of the electronic imaging sensor. Altematively or additionally, the user- 
20 operated imaging fimctionality selection switch is a mechanically responsive switch 
which selectably mechanically associates selected filters with the electronic imaging 
sensor. 

In accordance with still another preferred embodiment of the present 
invention the user-operated imaging fimctionality selection switch is an electronically 

25 responsive switch which selectably electronically determines responsiveness to selected 
outputs from predetermined portions of the electronic imaging sensor. 

In accordance with another preferred embodiment of the present 
invention the electronic camera also includes an array of red, green, blue and infrared 
passing pixelwise elements arranged over the electronic imaging sensor. Additionally or 

30 altematively, the electronic camera also includes a data entry template projector 
employed by the second imaging fimctionality. Preferably, the electronic camera also 
includes an illuminator employed by the second imaging ftmctionality. Additionally or 
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alternatively, the electronic camera also includes a synchronized illumination power 

variation functionality. 

In accordance with another preferred embodiment of the present 

invention the portable telephone also includes an array of red, green, blue and infrared 
5 passing pixelwise elements arranged over the electronic imaging sensor. Additionally or 

altematively, the portable telephone also includes a data entry template projector 

employed by the second imaging functionality. Preferably, the portable telephone also 

includes an illuminator employed by the second imaging functionality. Additionally or 

altematively, the portable telephone also includes a synchronized illimiination power 
10 variation functionality. 

In accordance with another preferred embodiment of the present 

invention the digital personal assistant also includes an array of red, green, blue and 

infrared passing pixelwise elements arranged over the electronic imaging sensor. 

Additionally or altematively, the digital personal assistant also includes a data entry 
15 template projector employed by the second imaging functionality. Preferably, the digital 

personal assistant also includes an illuminator employed by the second imaging 

functionality. Additionally or altematively, the digital personal assistant also includes a 

synchronized illumination power variation functionality. 

There is also provided in accordance with another preferred embodiment 
20 of the present invention an electronic camera including an electronic imaging sensor 

providing an output representing an imaged field and an array of red, green, blue and 

infrared passing pixelwise elements arranged over the electronic imaging sensor. 

There is further provided in accordance with still another preferred 

embodiment of the present invention a portable telephone including telephone 
25 functionality, an electronic imaging sensor providing an output representing an imaged 

field and an array of red, green, blue and infrared passing pixelwise elements arranged 

over the electronic imaging sensor. 

There is also provided in accordance with another preferred embodiment 

of the present invention a digital personal assistant including at least one personal 
30 digital assistant functionality, an electronic imaging sensor providing an output 

representing an imaged field and an array of red, green, blue and infrared passing 

pixelwise elements arranged over the electronic imaging sensor. 
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There is even further provided in accordance with yet another preferred 
embodiment of the present invention an electronic camera including an electronic 
imaging sensor providing an output representing an imaged field, a first imaging 
functionality employing the electronic imaging sensor in a visible radiation imaging 
mode for taking a picture of a scene in the imaged field and a second imaging 
functionality employing the electronic imaging sensor in an infrared radiation sensing 
mode for data entry responsive to user hand activity. 

There is also provided in accordance with another preferred embodiment 
of the present invention a portable telephone including telephone functionality, an 
electronic imaging sensor providing an output representing an imaged field, a first 
imaging functionality employing the electronic imaging sensor in a visible radiation 
imaging mode for taking a picture of a scene in the imaged field and a second imaging 
functionality employing the electronic imaging sensor in an infi*ared radiation sensing 
mode for data entry responsive to user hand activity. 

There is further provided in accordance with yet another preferred 
embodiment of the present invention a digital personal assistant including at least one 
personal digital assistant functionality, an electronic imaging sensor providing an output 
representing an imaged field, a first imaging functionality employing the electronic 
imaging sensor in a visible radiation imaging mode for taking a picture of a scene in the 
imaged field and a second imaging functionality employing the electronic imaging 
sensor in an infrared radiation sensing mode for data entry responsive to user hand 
activity. 

In accordance with another preferred embodiment of the present 
invention the electronic camera includes an array of red, green, blue and infi-ared 
passing pixelwise elements arranged over the electronic imaging sensor. 

In accordance with yet another preferred embodiment of the present 
invention the portable telephone includes an array of red, green, blue and infi-ared 
passing pixelwise elements arranged over the electronic imaging sensor. 

In accordance with yet another preferred embodiment of the present 
invention the digital personal assistant includes an array of red, green, blue and infrared 
passing pixelwise elements arranged over the electronic imaging sensor. 

Preferably, in the first imaging functionality output of the pixels 



underlying the red, green and blue passing elements are employed. Additionally, output 
of the pixels underlying the infrared passing elements is employed to prevent distortion 
of color response of the imaging sensor. Additionally, in the second imaging 
functionality output of the pixels underlying the infrared passing elements are 
5 employed. 

In accordance with yet another preferred embodiment of the present 
invention the electronic camera also includes a processor. In accordance with still 
another preferred embodiment of the present invention the electronic camera forms part 
of a telephone. Altematively, the electronic camera forms part of a personal digital 

10 assistant. Altematively, the electronic camera forms part of a wristwatch. 

In accordance with another preferred embodiment of the present 
invention the portable telephone also includes a processor. 

In accordance with yet another preferred embodiment of the present 
invention the digital personal assistant also includes a processor. 

15 In accordance with still another preferred embodiment of the present 

invention the processor includes the following functionality: as each pixel value is 
acquired, determining, using pixel coordinates, whether that pixel lies within a 
predefined keystroke region, acquiring pixel values for the pixel coordinates lying 
within the predefined keystroke region, adding or subtracting each of the pixel values to 

20 or from a pixel total maintained for each the keystroke region based on determining a 
pixel function of each pixel, comparing the pixel total for each the keystroke region 
with a current key actuation threshold, if the pixel total exceeds the key actuation 
threshold for a given keystroke region in a given frame and in the previous frame the 
pixel total did not exceed the key actuation threshold for that keystroke region, 

25 providing a key actuation output and if the pixel total does not exceed the key actuation 
threshold for a given keystroke region in a given frame and in the previous frame the 
pixel total did exceed the key actuation threshold for that keystroke region, providing a 
key deactuation output. 

In accordance with another preferred embodiment of the present 

30 invention the determining whether that pixel lies within a predefined keystroke region is 
made by employing a pixel index table which indicates for each pixel, whether that 
pixel lies within a predetermined keystroke region and, if so, within which keystroke 
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region it lies. Alternatively, both of the determining steps employ the pixel index table. 

In accordance with yet another preferred embodiment of the present 
invention the pixel total is maintained for each keystroke region in a keystroke region 
accumulator table. 

5 In accordance with still another preferred embodiment of the present 

invention the comparing employs a keystroke region threshold table. 

In accordance with another preferred embodiment of the present 
invention the processor also includes the following functionality: once all of the pixels 
in a ftume have been processed, determining an updated background level for a frame 

10 and determining a key actuation threshold for the keystroke region threshold table by 
subtracting the updated background level from a predetermined threshold level which is 
established for each keystroke region. Additionally, the pixel ftmction includes adding 
the pixel values of a plurality of pixels in the keystroke region. 

In accordance with yet another preferred embodiment of the present 

15 invention the processor is operative to determine the "center of gravity" of pixel values 
of pixels in the two-dimensional image sensor. Preferably, the processor includes the 
following fimctionality: as each pixel value is acquired, determining, using the pixel 
coordinates, whether that pixel lies within a predefined active region, acquiring pixel 
values for various pixel coordinates and determining the "center of gravity" of the pixel 

20 values. 

In accordance with still another preferred embodiment of the present 
invention determining the "center of gravity" is achieved by multiplying the pixel 
values by X and Y values representing the geographic position of each pixel, summing 
the results along mutually perpendicular axes X and Y, summing the total of the pixel 
25 values for all relevant pixels for the active region and dividing the siunmed results by 
the total of the pixel values to determine the X and Y coordinates of the "center of 
gravity", which represents a desired engagement location. 

Preferably, the pixel values are thresholded prior to summing thereof 
There is yet ftirther provided in accordance with another preferred 
30 embodiment of the present invention a wristwatch including wristwatch fimctionality, 
an electronic imaging sensor providing an output representing an imaged field, a first 
imaging functionality employing the electronic imaging sensor for taking a picture of a 
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scene in the imaged field, a second imaging functionality employing the electronic 
imaging sensor for data entry responsive to user hand activity and a user-operated 
imaging functionality selection switch operative to enable a user to select operation in 
one of the first and second imaging functionalities. 

There is also provided in accordance with another preferred embodiment 
of the present invention a vehicle including a jSrame, a drive train and a chassis, 
including at least one lockable entry portal and portal imlocking functionality and an 
access control assembly for governing authorized access to the at least one lockable 
entry portal including at least one at least partially light transmissive user finger 
engagement surface accessible fi-om outside the chassis, an illuminator operative to 
illuminate at least one user finger engagement plane adjacent the at least one user finger 
engagement surface, a two-dimensional imaging sensor viewing the at least one user 
finger engagement surface, fi-om a location inwardly of the at least one finger 
engagement surface, for sensing light fi-om the illuminator scattered by engagement of a 
user's finger with the at least one finger engagement surface, and a data entry processor 
receiving an output fi-om the two-dimensional imaging sensor and providing a data entry 
input to the portal unlocking functionality. 

There is further provided in accordance with yet another preferred 
embodiment of the present invention access control apparatus for use with a vehicle 
including a chassis having at least one lockable entry portal and portal unlocking 
functionality and also including at least one at least partially light transmissive user 
finger engagement surface accessible from outside the chassis, the access control 
apparatus including an illuminator operative to illuminate the at least one user finger 
engagement surface, a two-dimensional imaging sensor viewing the at least one user 
finger engagement surface from a location inwardly of the at least one finger 
engagement surface for sensing light from the illuminator scattered by engagement of a 
user's finger with the at least one finger engagement surface and a data entry processor 
receiving an output from the two-dimensional imaging sensor and providing a data entry 
input to the portal unlocking functionality. 

There is further provided in accordance with yet another preferred 
embodiment of the present invention access control apparatus for use with an enclosure 
having at least one lockable entry portal and portal unlocking functionality and also 
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including at least one at least partially light transmissive user finger engagement surface 
accessible from outside the enclosure, the access control apparatus including an 
illuminator operative to illuminate the at least one user finger engagement surface, a 
two-dimensional imaging sensor viewing the at least one user finger engagement 
5 surface from a location inwardly of the at least one finger engagement surface for 
sensing light fi-om the illuminator scattered by engagement of a user's finger with the at 
least one finger engagement surface and a data entry processor receiving an output fi-om 
the two-dimensional imaging sensor and providing a data entry input to the portal 
unlocking fimctionality. 

10 There is yet further provided in accordance with still another preferred 

embodiment of the present invention data entry apparatus for use with at least one at 
least partially light transmissive user finger engagement surface, the data entry 
apparatus including an illuminator operative to illuminate the at least one user finger 
engagement surface, a two-dimensional imaging sensor viewing the at least one user 

15 finger engagement surface fi-om a location inwardly of the at least one finger 
engagement surface for sensing light firom the illiraiinator scattered by engagement of a 
user's finger with the at least one finger engagement surface and a data entry processor 
receiving an output from the two-dimensional imaging sensor and providing a data entry 
input. 

20 Preferably, the illuminator is located at a location inwardly of the at least 

one engagement surface. Additionally or alternatively, the at least one finger 
engagement surface includes a one-dimensional array of a plurality of finger 
engagement surfaces. Alternatively, the at least one finger engagement surface includes 
a two-dimensional array of a plurality of finger engagement surfaces. 

25 In accordance with another preferred embodiment of the present 

invention the vehicle also includes an illumination director cooperating with the 
illuminator for providing an illumination beam generally parallel to the at least one 
finger engagement surface. 

In accordance with yet another preferred embodiment of the present 

30 invention the access control apparatus also includes an illumination director cooperating 
with the illuminator for providing an illumination beam generally parallel to the at least 
one finger engagement surface. 
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In accordance with still another preferred embodiment of the present 
invention the data entry apparatus also includes an illumination director cooperating 
with the illuminator for providing an illumination beam generally parallel to the at least 
one finger engagement surface. 
5 Preferably, the illumination director includes a prism. 

There is also provided in accordance with another preferred embodiment 
of the present invention data entry apparatus for use with at least one engagement 
siu*face, the data entry apparatus including a light emitting data entry engagement 
element, which emits light only when it is in at least predetermined propinquity to the at 
10 least one engagement surface, a two-dimensional imaging sensor viewing the at least 
one engagement surface for sensing light emitted by the light emitting data entry 
engagement element and a data entry processor receiving an output fi"om the two- 
dimensional imaging sensor and providing a data entry input. 

Preferably, the light emitting data entry engagement element includes an 
15 elongated element having an LED at an end thereof. Additionally, the LED includes an 
IR emitting LED. Additionally or altematively, the data entry apparatus also includes a 
proximity switch. 

There is further provided in accordance with yet another preferred 
embodiment of the present invention data entry apparatus for use with at least one 

20 engagement surface, the data entry apparatus including a projector operative to 
illuminate the at least one engagement surface and to define thereon at least one 
touchpad region and at least one keyboard region, the touchpad region being defined by 
a zero'th order diffi-acted image, an illuminator operative to illuminate the at least one 
engagement surface, a two-dimensional imaging sensor viewing the at least one 

25 engagement surface from a location inwardly of the at least one engagement surface for 
sensing light from the illuminator scattered by engagement of a user's finger with the at 
least one engagement surface and a data entry processor receiving an output from the 
two-dimensional imaging sensor and providing a data entry input. 

Preferably, the processor includes the following fimctionality: as each 

30 pixel value is acquired, determining, using pixel coordinates, whether that pixel lies 
within a predefined keystroke region, acquiring pixel values for the pixel coordinates 
lying within the predefined keystroke region, adding or subtracting each of the pixel 
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values to or from a pixel total maintained for each the keystroke region based on 
determining a pixel function of each pixel, comparing the pixel total for each the 
keystroke region with a current key actuation threshold, if the pixel total exceeds the 
key actuation threshold for a given keystroke region in a given frame and in the 
previous frame the pixel total did not exceed the key actuation threshold for that 
keystroke region, providing a key actuation output and if the pixel total does not exceed 
the key actuation threshold for a given keystroke region in a given frame and in the 
previous frame the pixel total did exceed the key actuation threshold for that keystroke 
region, providing a key deactuation output. 

There is even further provided in accordance with still another preferred 
embodiment of the present invention a data input device including an illuminator 
operative to illuminate at least one engagement plane by directing light along the at least 
one engagement plane, a two-dimensional imaging sensor, including an optical wedge 
element, viewing the at least one engagement plane from a location outside the at least 
one engagement plane for sensing light from the illuminator scattered by engagement of 
a data entry object with the at least one engagement plane and a data entry processor 
receiving an output from the two-dimensional imaging sensor and providing a data entry 
input to utilization circuitry. 

Preferably, the optical wedge element enables the imaging sensor to 
efficiently sense light from the engagement plane. 

There is still further provided in accordance with yet another preferred 
embodiment of the present invention a data input device including an illuminator 
operative to illuminate at least one engagement plane by directing light along the at least 
one engagement plane, a two-dimensional imaging sensor, including a pair of 
oppositely oriented optical wedge elements, viewing the at least one engagement plane 
from a location outside the at least one engagement plane for sensing light from the 
illuminator scattered by engagement of a data entry object with the at least one 
engagement plane and a data entry processor receiving an output from the two- 
dimensional imaging sensor and providing a data entry input to utilization circuitry. 

There is also provided in accordance with still another preferred 
embodiment of the present invention a data input device including an illuminator 
operative to illuminate at least one engagement plane by directing Hght along the at least 
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one engagement plane and to illuminate a proper positioning confirmation location on 
an engagement surface, a two-dimensional imaging sensor, including, viewing the at 
least one engagement plane firom a location outside the at least one engagement plane 
for sensing light from the illuminator scattered by engagement of a data entry object 
with the at least one engagement plane and a data entry processor receiving an output 
from the two-dimensional imaging sensor and providing a data entry input to utilization 
circuitry. 

Preferably, the imaging sensor senses the presence or absence of 
illumination at the proper positioning confirmation location. Additionally, the data entry 
apparatus is operative to unpower itself when the imaging sensor senses the absence of 
illumination at the proper positioning confirmation location. 

There is fiirther provided in accordance with yet another preferred 
embodiment of the present invention gaming apparatus including at least one game 
piece, a game template projector, operative to project a game template onto a surface, an 
illimiinator operative to illiuninate the siuface and an imaging sensor viewing the 
surface for sensing light from the illuminator reflected by at least one optically encoded 
reflector mounted on the at least one game piece. 

In accordance with another preferred embodiment of the present 
invention each of the at least one reflector identifies one of the at least one game piece 
and is operative to indicate a location of the at least one game piece on the game 
template. Additionally or alternatively, the at least one reflector also identifies a spatial 
orientation of the game piece. 
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BRIEF DESCRIPTION OF THE DRAWINGS 



The present invention will be understood and appreciated more fully 
from the following detailed description, taken in conjunction with the drawings in 
which: 

Figs. lA & IB are simplified pictorial illustrations of an electronic 
camera constructed and operative in accordance with a preferred embodiment of the 
present invention in two different modes of operation; 

Figs. 2A & 2B are simplified pictorial illustrations of a camera-equipped 
cellular telephone, constructed and operative in accordance with a preferred 
embodiment of the present invention in two different modes of operation; 

Figs. 3A & 3B are simplified pictorial illustrations of a camera-equipped 
personal digital assistant, constructed and operative in accordance with a preferred 
embodiment of the present invention in two different modes of operation; 

Figs. 4A & 4B are simpUfied diagrammatic illustrations of the operation 
of imaging fimctionahty selection in accordance with a preferred embodiment of the 
present invention in two different modes of operation; 

Figs. 5 A & 5B are simplified diagrammatic illustrations of the operation 
of imaging fimctionahty selection in accordance with another preferred embodiment of 
the present invention in two different modes of operation; 

Fig. 6 is a simphfied diagrammatic illustration of a data entry object 
engagement location sensing subsystem usefiil in the embodiments of any of Figs. I A - 
5B; 

Fig. 7 is a simplified flow chart illustrating operation of the data entry 
object engagement location sensing subsystem of Fig, 6 in accordance with a preferred 
embodiment of the present invention; 

Fig. 8 is a simpUfied illustration of a preferred data structure employed in 
the operation of the data entry object engagement location sensing subsystem shown in 
Fig. 7; 

Fig. 9 is a simplified pictorial illustration of outlines of typical keystroke 
regions as sensed by a two-dimensional image sensor viewing a keyboard, such as the 
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keyboard seen in Fig. 6; 

Fig. 10 is a simplified pictorial illustration of outlines of typical 
footprints of a typical light pattern occasioned by data entry object engagement with 
several keystroke regions, such as those shown in Fig. 9; 

Figs. 11 A, IIB and IIC are simplified illustrations of three alternative 
methodologies for determining the function of the pixel within the keystroke region in 
which it Hes as shown in Fig. 7; 

Figs. 12 A, 12B and 12C are simpUfied illustrations of traces which are 
useful in understanding Figs. 1 1 A, 1 IB and 1 IC; 

Fig. 13 is a simplified flow chart illustrating operation of a data entry 
object engagement location sensing subsystem employed in a tracking system and 
methodology constructed and operative in accordance with a preferred embodiment of 
the present invention; 

Fig. 14 is a simplified illustration showing synchronized illumination 
power variation fimctionality usefiil in accordance with a preferred embodunent of the 
present invention; 

Fig. 15 is a simpUfied partially pictorial, partially diagrammatic 
illustration of data entry apparatus embodied in a vehicle access control system; 

Fig. 16 is a simplified partially pictorial, partially diagrammatic 
illustration of a variation in the data entry apparatus of Fig, 15; 

Fig. 17 is a simplified pictorial illustration of data entry apparatus 
constructed and operative in accordance with another preferred embodiment of the 
present invention; 

Figs. 18A and 18B are respectively a simplified pictorial and a simpUfied 
sectional illustration of data entry apparatus constructed and operative in accordance 
with another preferred embodiment of the present invention; 

Figs. 19A and 19B are respectively a simplified pictorial and a simpUfied 
sectional illustration of data entry apparatus constmcted and operative in accordance 
with yet another preferred embodiment of the present invention; 

Figs. 20A and 20B are respectively a simpUfied pictorial and a simplified 
sectional illustration of data entry apparatus constmcted and operative in accordance 
with still another preferred embodiment of the present invention; 
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Fig. 21 is a simplified pictorial illustration of data entry apparatus 
constructed and operative in accordance with a further preferred embodiment of the 
present invention; 

Fig. 22 is a simplified pictorial illustration of data entry apparatus 
constructed and operative in accordance with another preferred embodiment of the 
present invention; 

Fig, 23 is a simplified pictorial illustration of gaming apparatus 
constructed and operative in accordance with a preferred embodiment of the present 
invention; and 

Fig. 24 is a simplified pictorial illustration of gaming apparatus 
constructed and operative in accordance with another preferred embodiment of the 
present invention. 
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DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS 



Reference is now made to Figs. 1 A & IB, which are simplified pictorial 
illustrations of an electronic camera constructed and operative in accordance with a 
preferred embodiment of the present invention in two different modes of operation. As 
seen in Figs. lA & IB, an electronic camera 100, which may be of conventional 
construction other than as specifically described hereinbelow, such as a Nikon 
COOLPIX 5700, available fi-om Nikon Corporation, Tokyo, Japan, is equipped with an 
electronic imaging sensor module 102, which may include a CMOS array, for example, 
CMOS array OVT6130, available fi-om Omnivision Technologies, Inc., 1341 Orleans 
Drive, Simnyvale, CA, USA, providing an output representing an imaged field. 

A user-operated imaging fimctionality selection switch 104 is preferably 
provided to enable a user to select operation in one of two possible imaging 
fimctionalities. In a first imaging functionality, illustrated in Fig. lA, the electronic 
imaging sensor module 102 is employed to take a picture of a scene in the imaged field. 
In a second imaging fimctionaUty, illustrated in Fig. IB, the electronic imaging sensor 
module 102 is employed for data entry responsive to user hand activity. 

Various embodiments of the first and second imaging fimctionalities will 
be described hereinbelow, with reference to Figs. 4A-5B. Preferably, but not 
necessarily, the second imaging fimctionality, as shown in Fig, IB, employs a data entry 
template projector 106, which projects a data entry template, such as that designated by 
reference numeral 108, onto a surface, such as a table surface 110. An illuminator 111 
preferably is provided and is operative to illuminate a region adjacent the data entry 
template 108. The imaging sensor module 102 views the data entry template region for 
sensing light fi-om the illuminator 111. The second imaging fimctionaUty also preferably 
incorporates processing fimctionality, an example of which is described in applicant's 
Published PCT Application WO 02/054169 A2, the disclosure of which is hereby 
incorporated by reference. 

It is seen in Fig. 1 A that when the electronic camera 100 is operative in 
the first imaging fimctionality, also termed a pictxire mode, a digital picture 112 is 
provided. Similarly, it is seen in Fig. IB that when the electronic camera 100 is 
operative in the second imaging functionahty, also termed a data mode, a stream of data 
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114 is provided. 

Reference is now made to Figs. 2A & 2B, which are simplified pictorial 
illustrations of a camera-equipped cellular telephone, constructed and operative in 
accordance with a preferred embodiment of the present invention in two different modes 
5 of operation. As seen in Figs. 2A & 2B, a camera-equipped cellular telephone 200, 
which may be of conventional construction other than as specifically described 
hereinbelow, such as a Samsung SPH-I330, available from Samsung Electronics, Seoul, 
Korea, is equipped with an electronic imaging sensor module 202, which may include a 
CMOS array, for example, CMOS array OVT6130, available from Omni vision 

10 Technologies, Inc., 1341 Orleans Drive, Sunnyvale, CA, USA, providing an output 
representing an imaged field. 

A user-operated imaging functionality selection switch 204 is preferably 
provided to enable a user to select operation in one of two possible imaging 
fimctionaHties. In a first imaging functionality, illustrated in Fig. 2A, the electronic 

15 imaging sensor module 202 is employed to take a picture of a scene in the imaged field. 
In a second imaging functionality, illustrated in Fig. 2B, the electronic imaging sensor 
module 202 is employed for data entry responsive to user hand activity. 

Various embodiments of the first and second imaging functionalities will 
be described hereinbelow, with reference to Figs. 4A-5B. Preferably, but not 

20 necessarily, the second imaging functionality, as shown in Fig. 2B, employs a data entry 
template projector 206, which projects a data entry template, such as that designated by 
reference numeral 208, onto a surface, such as a table surface 210. An illuminator 211 
preferably is provided and is operative to illuminate a region adjacent the data entry 
template 208. The imaging sensor module 202 views the data entry template region for 

25 sensing light from the illuminator 211. The second imaging functionality also preferably 
incorporates processing functionality an example of which is described in applicant's 
Pubhshed PCT AppUcation WO 02/054169 A2, the disclosure of which is hereby 
incorporated by reference. 

It is seen in Fig. 2A that when the camera-equipped cellular telephone 

30 200 is operative in the first imaging fimctionaUty, also termed a picture mode, a digital 
picture 212 is provided. Similarly, it is seen in Fig. 2B that when the camera-equipped 
cellular telephone 200 is operative in the second imaging fimctionaUty, also termed a 
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data mode, a stream of data 214 is provided. 

Reference is now made to Figs. 3A & 3B, which are simplified pictorial 
illustrations of a camera-equipped personal digital assistant, constructed and operative 
in accordance with a preferred embodiment of the present invention in two different 
5 modes of operation. As seen in Figs. 3A & 3B, a camera-equipped personal digital 
assistant 300, which may be of conventional construction other than as specifically 
described hereinbelow, such as a Samsung SPH-I700, available from Samsung 
Electronics, Seoul, Korea, is equipped with an electronic imaging sensor module 302, 
which may include a CMOS array, for example, CMOS array OVT6130, available from 

10 Omni vision Technologies, Inc., 1341 Orleans Drive, Sunnyvale, CA, USA, providing 
an output representing an imaged field. 

A user-operated imaging functionality selection switch 304 is preferably 
provided to enable a user to select operation in one of two possible imaging 
functionalities. In a first imaging functionality, illustrated in Fig. 3A, the electronic 

15 imaging sensor module 302 is employed to take a picture of a scene in the imaged field. 
In a second imaging functionality, illusfrated in Fig. 3B, the electronic imaging sensor 
module 302 is employed for data entry responsive to user hand activity. 

Various embodiments of the first and second imaging functionalities will 
be described hereinbelow, with reference to Figs. 4A-5B. Preferably, but not 

20 necessarily, the second imaging functionality, as shown in Fig. 3B, employs a data entry 
template projector 306, which projects a data entry template, such as that designated by 
reference numeral 308, onto a surface, such as a table surface 310, An illuminator 311 
preferably is provided and is operative to illuminate a region adjacent the data entry 
template 308. The imaging sensor module 302 views the data entry template region for 

25 sensing light from the illuminator 3 1 1 .The second imaging functionality also preferably 
incorporates processing functionality an example of which is described in applicant's 
Published PCT Application WO 02/054169 A2, the disclosure of which is hereby 
incorporated by reference. 

It is seen in Fig. 3A that when the electronic camera 300 is operative in 

30 the first imaging functionality, also termed a picture mode, a digital picture 312 is 
provided. Similarly, it is seen in Fig. 3B that when the electronic camera 300 is 
operative in the second imaging functionality, also termed a data mode, a stream of data 

18 



314 is provided. 

Reference is now made to Figs. 4 A & 4B, which are simpUfied 
diagrammatic illustrations of the operation of imaging functionality selection in 
accordance with a preferred embodiment of the present invention in two different modes 
5 of operation. As seen in Figs. 4A & 4B, one implementation of the invention involves 
the use of a first and second selectably positionable optical assemblies which may be 
selectably placed in front of an electronic imaging sensor module 402. 

In the illustrated embodiment, the sensor module 402 comprises a two- 
dimensional CMOS detector array 404 which is overlaid by an array of color filters 406, 
10 which pass infrared radiation. A lens 408 focuses light from an imaging field onto array 
404. 

Fig. 4A shows a first selectably positionable optical assembly 410, 
preferably comprising an IR blocking filter 412, aligned with lens 408. The imaging 
functionality of Fig. 4A is suitable for picture taking, since it prevents IR radiation from 

1 5 distorting the color response of the sensor module 402. 

Fig. 4B shows a second selectably positionable optical assembly 420, 
preferably comprising a visible radiation blocking filter 422, an IR band pass filter 424 
and a field lens 426, aligned with lens 408. The imaging functionality of Fig. 4B is 
suitable for data input, since it allows only a relatively narrow band of IR radiation to 

20 reach the sensor module 402 from a relatively wide field of view. 

Reference is now made to Figs. 5A & 5B, which are simplified 
diagrammatic illustrations of the operation of imaging functionality selection in 
accordance with another preferred embodiment of the present invention in two different 
modes of operation. In this embodiment, imaging functionality selection is effected 

25 electronically, without the need for mechanically movable optical elements. 

As seen in Figs. 5 A & 5B, this implementation of the invention involves 
the use of an electronic imaging sensor module 502 including a CMOS detector array 
504 overlaid with a repeating array of pixelwise selective light transmissive filters 506 
characterized in that the array includes red (R), green (G), blue (B) and infrared (IR) 

30 transmissive filters. A lens 508 focuses light from an imaging field onto detector array 
504. 

Fig. 5 A shows the imaging functionality of the embodiment of Figs. 5 A 
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and 5B in a picture taking mode, wherein only the outputs of the pixels underlying the 
R, G and B filters are employed and the respective IR content of the outputs of each of 
the pixels underlying the R, G and B filters is subtracted therefrom. This color 
processing prevents IR radiation from distorting the color response of the sensor module 
5 502. 

Fig. 5B shows the imaging functionality of the embodiment of Figs. 5 A 
and 5B in a data entry mode, wherein only the outputs of the pixels underlying the IR 
filters £ire employed. 

Reference is now made to Fig. 6, which is a simplified and generalized 

10 illustration of a projection keyboard system and methodology which is preferably 
incorporated into the embodiments of Figs. lA - 5B in accordance with a preferred 
embodiment of the present invention. A projection subsystem 550, preferably including 
a solid state light source (not shown) which illuminates a spatial light modulation 
element (not shown), defines an image 552 of a keyboard on a projection surface 554, 

15 preferably an inert surface, such as a desktop. 

An illumination subsystem 556, preferably including a solid state hght 
source (not shown), directs light in a radially directed illumination pattern 558, which 
preferably extends in a plane generally parallel to the projection surface 554. It is 
appreciated that the radially directed illumination pattern 558 has a very narrow spread 

20 in the direction generally perpendicular to the projection surface 554. It is further 
appreciated that the radially directed illumination pattern 558 is preferably located very 
close to the projection surface 554, 

Impingement of light in the radially directed illumination pattern 558 on 
a data entry object 560, such as a user's finger, a stylus or other implement, causes light 

25 to be scattered or reflected therefrom. It is appreciated that the light is only scattered or 
reflected when the data entry object 560 is in close contact with the keyboard 552 
defined on projection surface 554, 

A detection subsystem 562, preferably employing a solid state imaging 
sensor (not shown) incorporated in a camera in the embodiments of Figs. lA - 5B, 

30 receives light reflected or scattered from data entry object 560, The received light is 
mapped onto the imaging sensor with respect to a representation of the keyboard for 
associating the location of the data entry object 560 sensed by detection subsystem 562 
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with a key location 563 on the keyboard 552. Thus, the sensed location of data entry 
object 560 indicates which key of the keyboard 552 is being engaged. 

Reference is now made to Fig. 7, which is a simplified flow chart 
illustrating operation of a data entry object engagement location sensing subsystem of 
Fig. 6 in accordance with a preferred embodiment of the present invention, and to Fig. 
8, which is a simplified illustration of a preferred data structure employed in the 
operation of the data entry object engagement location sensing subsystem shown in Fig. 
7. 

Fig. 8 shows a simpUfied illustration of a preferred data structure 
employed in the operation of the data entry object engagement location sensing method 
described hereinbelow with respect to Fig. 7. It is appreciated that the imaging sensor of 
a camera, such as camera 100 (Fig. lA), is typically comprised of a set of M x N pixels, 
wherein a particular group of pixels views a defined region of the engagement plane 
which preferably overlies the projection surface 1 10 (Fig. 1 A). Thus, it is possible that a 
particular pixel group, located in the image plane of the camera 100, may receive 
scattered light fi-om a data entry object 560 (Fig. 6) engaging the key location 563 (Fig. 
6). 

Thus, as the camera 100 views the projection surface 1 10, each of the M 
X N pixels in the image plane of the camera 100 may receive light fi*om a corresponding 
region in the engagement plane in respect of a data entry object engagement therewith. 

As seen in Fig. 7, pixel values, such as gray level values, are acquired for 
various pixel coordinates. As each pixel value is acquired, a determination is made, 
using the pixel coordinates, as to whether that pixel lies within a predefined keystroke 
region, such as keystroke regions 600 shown in Fig. 9. This determination is preferably 
made by employing a pixel index table 601 which indicates, for each pixel, whether that 
pixel lies within a predetermined keystroke region, such as keystroke regions 605, 606, 
607 and 608 (Fig. 9), and, if so, within which keystroke region it lies. 

The fimction of the pixel within the keystroke region in which it lies is 
then determined, preferably by employing table 601. This fimction is typically additive 
or subtractive, but may altematively have another fimction. Typically, depending on the 
function, the pixel value is added to or subtracted from a pixel total maintained for each 
keystroke region in a keystroke region accumulator table 602. 
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Once all of the pixels in a frame have been processed as aforesaid, an 
updated background level is determined for the frame and a key actuation threshold is 
determined typically by subtracting the updated background level from a predetermined 
threshold level which is established for each keystroke region. This is preferably carried 
out by employing a keystroke region threshold table 604. 

The contents of the keystroke region accumulator table 602 for each 
keystroke region preferably are then compared with the current key actuation threshold. 
If the contents of the accumulator table 602 exceed the key actuation threshold for a 
given key actuation region in a given frame and in the previous frame the contents of 
the accumulator table 602 did not exceed the key actuation threshold, a key actuation 
output is provided. 

Similarly, if the contents of the accumulator table 602 do not exceed the 
key actuation threshold for a given key actuation region in a given frame and in the 
previous frame the contents of the accumulator table 602 did exceed the key actuation 
threshold, a key deactuation output is provided. In all other cases, no output need be 
generated. 

Reference is now made to Fig. 9, which is a simplified pictorial 
illustration of outlines of typical keystroke regions 605, 606, 607 and 608 as sensed by a 
two-dimensional image sensor (Fig. 6) viewing a keyboard, such as the keyboard 552, 
seen in Fig. 6. 

Reference is now made to Fig. 10, which is a simplified pictorial 
illustration of outlines of typical footprints 609, 610, 611 and 612 of a typical light 
pattem occasioned by data entry object engagement corresponding to the keystroke 
regions 605, 606, 607 and 608 (Fig. 9). 

Reference is now made to Figs. 11 A, IIB and UC, which are simplified 
illustrations of three alternative methodologies for determining the fimction of the pixel 
within the keystroke region in which it lies as shown in Fig. 8, and to Figs. 12A, 12B 
and 12C, which are simplified illustrations of traces which are usefiil in imderstanding 
Figs. 11 A, llBandllC. 

Turning now to Fig. 11 A, there is shown arranged along a common 
arbitrary axis 618 a simplified keystroke region 620 corresponding to a given key and 
containing a plurality of pixels 622. A typical simpUfied footprint of a typical light 
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pattern occasioned by data entry object engagement with the given key is indicated by 
reference numeral 624. Fig. 10 shows outlines of typical footprints 609, 610, 611 and 
612, corresponding to keystroke regions designated 605, 606, 607 and 608 in Fig. 9. 

A typical background signal pattem is indicated by reference numeral 
634. Superimposition of the footprint 624 over the background signal pattem 634 is 
indicated at reference numeral 635. A one dimensionally selectable overlap of footprint 
624 over keystroke region 620 is indicated at reference numeral 636. A one 
dimensionally selectable overlap of background signal pattem 634 over keystroke 
region 620 is indicated at reference numeral 637. A one dimensionally selectable 
overlap of superimposition 635 over keystroke region 620 is indicated at reference 
numeral 638. 

Fig. 12A illustrates a bias function 640 corresponding to a cross section 
of the keystroke region 620 taken along axis 618, which bias function defines keystroke 
region 620 along axis 618. There is also seen a signal function 644 corresponding to a 
cross section of the footprint 624 along axis 618; a backgrovmd signal function 646 
corresponding to a cross section of the background signal pattem 634 along axis 618 
and a combination signal 648 corresponding to a cross section of the superimposition 
635. 

There is also shown in Fig. 12A a one dimensional convolution 650 
corresponding to one dimensionally selectable overlap 636; a one dimensional 
convolution 652 corresponding to one dimensionally selectable overlap 637 and a one 
dimensional convolution 654 corresponding to one dimensionally selectable overlap 
638. Convolution 650 is shown with a threshold 660; convolution 652 is shown with a 
threshold 662 and convolution 654 is shown with a threshold 664. 

Turning now to Fig. IIB, there is shown arranged along a common 
arbitrary axis 670 a simplified keystroke region 680 corresponding to a given key and 
containing a plurality of pixels 682. The simplified keystroke region 680 is here shown 
surrounded by a simplified keystroke region border 683. A typical simphfied footprint 
of a typical light pattem occasioned by data entry object engagement with the given key 
is indicated by reference numeral 684. A typical backgroimd signal pattem is indicated 
by reference numeral 686. Superimposition of the footprint 684 over the background 
signal pattem 686 is indicated at reference numeral 688. A one dimensionally selectable 
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overlap of footprint 684 over keystroke region 680 and border 683 is indicated at 
reference numeral 690. A one dimensionally selectable overlap of background signal 
pattern 686 over keystroke region 680 and border 683 is indicated at reference numeral 
692. A one dimensionally selectable overlap of superimposition 688 over keystroke 
region 680 and border 683 is indicated at reference numeral 694. 

Fig. 12B illustrates a bias function 700 corresponding to a cross section 
of the keystroke region 680 and of the border 683 taken along axis 670, which bias 
function defmes keystroke region 680 along axis 670. It is seen that border 683 is 
assigned a negative value relative to the value of the keystroke region 680. This value 
assignment is appreciated to enhance the value of data entry object engagements to the 
extent that they lie within the keystroke region 680 and to decrease the value of such 
data entry object engagements to the extent that they extend over the border 683. There 
is also seen a signal function 704 corresponding to a cross section of the footprint 684 
along axis 670; a background signal function 706 corresponding to a cross section of the 
background signal pattem 686 along axis 670 and a combination signal 708 
corresponding to a cross section of the superimposition 688. 

There is also shown in Fig. 12B a one dimensional convolution 720, 
corresponding to one dimensionally selectable overlap 690; a one dimensional 
convolution 722, corresponding to one dimensionally selectable overlap 692 and a one 
dimensional convolution 724 corresponding to one dimensionally selectable overlap 
694, Convolution 720 is shown with a threshold 726; convolution 722 is shown with a 
threshold 727 and convolution 724 is shown with a threshold 728. 

Turning now to Fig. IIC, there is shown arranged along a common 
arbitrary axis 730 a simplified keystroke region 740 corresponding to a given key and 
containing a plurality of pixels 741. The simpUfied keystroke region 740 is here shown 
surrounded by a simpUfied keystroke region inner border 742 and by a simplified 
keystroke region outer border 743. A typical simpUfied footprint of a typical light 
pattem occasioned by data entry object engagement with the given key is indicated by 
reference numeral 744. A typical background signal pattem is indicated by reference 
numeral 746. Superimposition of the footprint 744 over the background signal pattem 
746 is indicated at reference numeral 748. A one dimensionally selectable overlap of 
footprint 744 over keystroke region 740 and borders 742 and 743 is indicated at 
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reference numeral 750. A one dimensionally selectable overlap of background signal 
pattern 746 over keystroke region 740 and borders 742 and 743 is indicated at reference 
numeral 752. A one dimensionally selectable overlap of superimposition 748 over 
keystroke region 740 and borders 742 and 743 is indicated at reference numeral 754. 
5 Fig. 12C illustrates a bias function 760 corresponding to a cross section 

of the keystroke region 740 and of the borders 742 and 743 taken along axis 730, which 
bias function defines keystroke region 740 along axis 730. It is seen that border 742 is 
assigned a zero value and border 743 is assigned a negative value relative to the value of 
the keystroke region 740. It is appreciated that these value assignments enhance the 

10 value of data entry object engagements that to the extent that they lie within the 
keystroke region 740 and to decrease the value of such data entry object engagements to 
the extent that they extend across the border 742 and at least into border 743. There is 
also seen a signal fimction 764 corresponding to a cross section of the footprint 744 
along axis 730; a backgroimd signal function 766 corresponding to a cross section of the 

15 background signal pattern 746 along axis 730 and a combination signal 768 
corresponding to a cross section of the superimposition 748. 

There is also shown in Fig. 12C a one dimensional convolution 780, 
corresponding to one dimensionally selectable overlap 750; a one dimensional 
convolution 782, corresponding to one dimensionally selectable overlap 752 and a one 

20 dimensional convolution 784 corresponding to one dimensionally selectable overlap 
754. Convolution 780 is shovm with a threshold 786; convolution 782 is shown with a 
threshold 787 and convolution 784 is shown with a threshold 788. 

It may be appreciated from a consideration of convolutions 638, 694 and 
754 that the dual border arrangement of Figs. 11 C and 12C provides the best detection 

25 of data entry object keystroke engagement, while minimizing backgroimd effects. 

Reference is now made to Fig. 13, which is a simpUfied flow chart 
illustrating operation of a data entry object engagement location sensing subsystem 
employed in a tracking system and methodology constructed and operative in 
accordance with a preferred embodiment of the present invention. 

50 As seen in Fig. 13, pixel values, such as gray level values, are acquired 

for various pixel coordinates. As each pixel value is acquired, a determination may be 
made, using the pixel coordinates, as to whether that pixel lies within a predefined 
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active region. Typically, if the pixel does not lie within a predetermined active region, 
its value is ignored. 

The pixel value for each pixel is preferably thresholded and typically all 
pixel values falling below a given threshold are ignored. The remaining pixel values 
may be weighted by a selected weighting parameter. In order to determine the "center of 
gravity" of the thresholded and weighted pixel values, the thresholded and weighted 
pixel values are multipHed respectively by X and Y values representing the coordinate 
position of each pixel and the results are summed along mutually perpendicular axes X 
and Y and stored in X and Y accumulators. The total of the thresholded and weighted 
pixel values for all relevant pixels are also summed and stored in a data accumulator, for 
the entire active region. 

Once all of the pixels in a frame have been processed as aforesaid, the 
summed thresholded and weighted pixel values along the X and Y axes respectively are 
divided by the total of the thresholded and weighted pixel values for the entire active 
region to determine the X and Y coordinates of the "center of gravity", which represents 
a desired engagement location. 

Reference is now made to Fig. 14, which is a simplified illustration 
showing synchronized illumination power variation fimctionality usefiil in accordance 
with a preferred embodiment of the present invention. The functionality illustrated in 
Fig. 14 is directed to modulating the amount of illimiination provided for data entry 
object engagement detection. This modulation is desirable because the intensity of light 
impinging on a data entry object and thus scattered thereby, decreases with the distance 
between an illiuninator 830 and a data entry object. Thus, it may be appreciated that the 
amount of light impinging on a data entry object 832 is substantially greater than the 
amount of light impinging on a data entry object 834. Furthermore, the amoimt of 
scattered light impinging on a detector 836 decreases with the distance between the data 
entry object and the detector. These two distance dependencies make detection of data 
entry object engagement difficult. 

In order to overcome this difficulty, there is provided in accordance with 
a preferred embodiment of the present invention variable intensity drive electronics 840 
which is coupled to both illuminator 830 and detector 836 and which causes the 
intensity of light produced by the illvmiinator 830 to vary, typically in a ramp fashion, in 
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synchronization to the imaging field location of detector 836. 

Thus, it may be seen that when a near portion (A) of the engagement 
surface 554 (Fig. 6) is being imaged, typically at the top portion A of detector 836, the 
intensity is at a minimum. When an intermediate portion (B) of the engagement surface 
554 is being imaged, typically at the middle of detector 836, the intensity is at an 
intermediate level and when a far portion (C) of the engagement surface 554 is being 
imaged, typically at the bottom portion (C) of the detector 836, the intensity is at a 
maximum. 

Variable intensity drive electronics 840 operates preferably by providing 
a synchronization output 842 to detector 836 and a corresponding synchronization 
output 844 to illuminator 830, causing the intensity level to increase in time in 
synchronization with the location of a scanned image region in detector 836. 

Reference is now made to Fig. 15, which is a simplified partially 
pictorial, partially diagrammatic illustration of data entry apparatus embodied in a 
vehicle access control system. As seen in Fig. 15, a vehicle 1500, typically comprising a 
firame, a drive train and a chassis including at least one lockable entry portal or door 
1502 and portal imlocking fimctionality, includes an access control assembly designated 
generally by reference numeral 1504. 

The access control assembly 1504 preferably includes at least one at least 
partially light transmissive user finger engagement surface accessible from outside the 
chassis. In the illustrated embodiment, plural user finger engagement surfaces 1506 are 
defined on a car door window 1508. An illuminator 1510, preferably located interiorly 
of the exterior of the vehicle, such as an IR line generator, is operative to illuminate a 
finger engagement plane 1512 adjacent the user finger engagement surfaces 1506. A 
preferred IR line generator is described in applicant's Published PCT Application 
WO02/054169 A2, the disclosure of which is hereby incorporated by reference. 

A two-dimensional imaging sensor 1514, located interiorly of the 
exterior of the vehicle, views the finger engagement surfaces 1506, from a location 
inwardly of the car door window 1508, for sensing light originating from the illuminator 
1510, which is scattered by engagement of a user's finger with the finger engagement 
surfaces 1506. A preferred two-dimensional imaging sensor 1514 is described in 
appHcant's Published PCT Application WO02/054169 A2, the disclosure of which is 
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hereby incorporated by reference. 

A data entry processor 1516 receives an output from the two-dimensional 
imaging sensor 1514 and provides a data entry input to a conventional door unlocking 
functionality (not shown). 
5 Reference is now made to Fig. 16, which is a simplified diagrammatic 

illustration of a variation in the data entry apparatus of Fig. 15. As seen in Fig. 16, a 
vehicle 1600, typically comprising a frame, a drive train and a chassis including at least 
one lockable entry portal or door 1602 and portal unlocking functionality, includes an 
access control assembly designated generally by reference numeral 1604. 

10 The access control assembly 1604 preferably includes at least one at least 

partially light transmissive user finger engagement surface accessible from outside the 
chassis. In the illustrated embodiment, plural user finger engagement surfaces 1606 are 
defined on a car door window 1608. In this embodiment, the plural user finger 
engagement surfaces 1606 may be arranged in a two-dimensional array. An illuminator 

15 1610, preferably located interiorly of the exterior of the vehicle, such as an IR line 
generator, is operative to illuminate a finger engagement plane 1612 adjacent the user 
finger engagement surfaces 1606. A preferred IR line generator is described in 
applicant's Published PCT Application WO02/054169 A2, the disclosure of which is 
hereby incorporated by reference. 

20 In the embodiment of Fig. 16, the finger engagement planes 1612 are 

parallel to the finger engagement surfaces 1606, by virtue of provision of a prism 1613 
disposed typically on the outside of the vehicle window 1608 along the light path from 
illuminator 1610. 

A two-dimensional imaging sensor 1614, located interiorly of the 
25 exterior of the vehicle, views the finger engagement surfaces 1606, from a location 
inwardly of the car door window 1608, for sensing light originating from the illuminator 
1610, which is scattered by engagement of a user's finger with the finger engagement 
surfaces 1606. A preferred two-dimensional imaging sensor 1614 is described in 
applicant's Published PCT Application WO02/054169 A2, the disclosure of which is 
30 hereby incorporated by reference. 

A data entry processor 1616 receives an output from the two-dimensional 
imaging sensor 1614 and provides a data entry input to a conventional door unlocking 
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functionality (not shown). 

It is appreciated that portions of the apparatus of Figs. 15 and 16 may 
find application in other types of access control environments and in various data entry 
envirormients. 

Reference is now made to Fig. 17, which is a simplified pictorial 
illustration of data entry apparatus constructed and operative in accordance with another 
preferred embodiment of the present invention. As seen in Fig. 17, a light emitting data 
entry engagement element, preferably in the form of a stylus 1700, emits light when and 
only when it is in at least predetermined propinquity to an engagement surface 1702. 

A two-dimensional imaging sensor 1704 views the engagement surface 
1702 for sensing light emitted by the stylus 1700. A data entry processor 1706 receives 
an output fi-om the two-dimensional imaging sensor 1704 and provides a data entry 
input. 

As seen in Fig. 17, the stylus 1700 preferably comprises an IR emitting 
LED 1708 located in an IR transmissive tip 1710 thereof. A proximity switch 1712, 
such as a contact switch, provides an output indication of the proximity of tip 1710 to 
engagement surface 1702. Circuitry 1714 receives an input firom proximity switch 1712 
and operates the LED 1708 accordingly, so as to provide an IR emission whose location 
indicates the location of stylus impingement with engagement surface 1702, 

Reference is now made to Figs. 18 A and 18B, which are respectively a 
simplified pictorial and a simplified sectional illustration of data entry apparatus 
constructed and operative in accordance with another preferred embodiment of the 
present invention. As seen in Figs. ISA & 18B, the data entry apparatus comprises a 
projector 1800 operative to illuminate at least one engagement surface 1802 and to 
define thereon at least one touchpad region 1804 and at least one keyboard region 1806, 
the touchpad region 1804 being defined by a zero'th order diffracted image provided by 
the projector 1800. 

Turning particularly to Fig. 18B, it is seen that the projector 1800 
comprises a light source, such as a diode laser 1810, which outputs via a coUimating 
lens 1812 onto a first diflfractive optical element 1814. Diffiractive optical element 1814 
diffracts the light impinging thereon into a primary keyboard image beam 1816 which 
impinges on at last one keyboard region 1806 and into a secondary ghost image beam 
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1818, which is preferably blocked by a barrier 1819. Residual light passing through 
dififractive optical element 1814 defines a zero'th order beam 1820, which impinges 
onto a second diffractive optical element 1822. 

Diffractive optical element 1822 diffracts the light impinging thereon 
5 from the zero'th order beam 1820 into a touchpad defining beam 1824 which impinges 
on at least one touchpad region 1804. 

Returning to Fig. 18 A, it is seen that there is also provided an illuminator 
1830, operative to illuminate the finger engagement surface. A two-dimensional 
imaging sensor 1832 views the engagement surface for sensing light from the 

10 illimiinator 1830 scattered by engagement of a user's finger with the finger engagement 
surface. A data entry processor 1834 receives an output from the two-dimensional 
imaging sensor 1832 and provides a data entry input. 

Reference is now made to Figs. 19A and 19B, which are respectively a 
simplified pictorial and a simplified sectional illustration of data entry apparatus 

15 constmcted and operative in accordance with yet another preferred embodiment of the 
present invention. As seen in Figs. 19A & 19B, the data entry apparatus comprises a 
projector 1900 operative to illuminate at least one finger engagement surface 1902 and 
to define thereon typically at least one keyboard region 1904. An illuminator 1906 is 
operative to illuminate the finger engagement surface 1902. 

20 An imaging sensor module 1908, including a two-dimensional imaging 

sensor 1910, views the engagement surface for sensing light from the illuminator 1906 
scattered by engagement of a user's finger with the finger engagement surface 1902. A 
particular feature of the imaging sensor module 1908 is that it includes an optical wedge 
element 1912 which provides a tilt, enabling a generally vertically disposed imaging 

25 sensor 1910 to efficiently sense light from a finger engagement surface 1902 extending 
generally perpendicularly thereto. 

A data entry processor 1914 receives an output from the two-dimensional 
imaging sensor 1910 and provides a data entry input. 

Reference is now made to Figs. 20A and 20B, which are respectively a 

30 simplified pictorial and a simplified sectional illustration of data entry apparatus 
constructed and operative in accordance with still another preferred embodiment of the 
present invention. As seen in Figs. 20A & 20B, the data entry apparatus comprises a 
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projector 2000 operative to illuminate at least one finger engagement surface 2002 and 
to define thereon typically at least one keyboard region 2004. An illuminator 2006 is 
operative to illuminate the finger engagement surface 2002. 

An imaging sensor module 2008, including a two-dimensional imaging 
sensor 2010, views the engagement surface for sensing light fi-om the illuminator 2006 
scattered by engagement of a user's finger with the finger engagement surface 2002. A 
particular feature of the imaging sensor module 2008 is that it includes a pair of 
oppositely oriented optical wedge elements 2012 and 2014, each of which provides a tilt 
to a portion of the light incoming to sensor 2010, enabling efficient use of area of 
generally rectangular imaging sensor, as seen at reference number 2016. 

Reference is now made to Fig. 21, which is a simplified pictorial 
illustration of data entry apparatus constructed and operative in accordance with yet 
another preferred embodiment of the present invention. As seen in Fig. 21, the data 
entry apparatus comprises a projector 2100 operative to illuminate at least one finger 
engagement surface 2102 and to define thereon typically at least one keyboard region 
2104. An illuminator 2106 is operative to illuminate the finger engagement surface 
2102. 

An imaging sensor module 2108, including a two-dimensional imaging 
sensor 2110, views the engagement surface for sensing light fi-om the illuminator 2106 
scattered by engagement of a user's finger with the finger engagement surface 2102. 

A particular feature of the illuminator 2106 is that defines an auxiliary 
light path 2112 which impinges on a proper positioning confirmation location 2114 on 
the finger engagement surface 2102 adjacent and preferably nearer to the illuminator 
than the at least one keyboard region 2104. The imaging sensor module 2108 senses the 
presence of illumination at location 2114 and in the absence of sensed illumination at 
that location, typically indicating improper placement of the data entry apparatus, 
typically unpowers the data entry apparatus. 

A data entry processor 2116 receives an output firom the two-dimensional 
imaging sensor 2110 and provides a data entry input. 

Reference is now made to Fig. 22, which is a simplified pictorial 
illustration of data entry apparatus constmcted and operative in accordance with another 
preferred embodiment of the present invention in the form of a wristwatch. As seen in 
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Fig. 22, a wristwatch 2200, which may be of conventional construction other than as 
specifically described hereinbelow, is equipped with an electronic imaging sensor 
module 2202, which may include a CMOS array, for example, CMOS array OVT6130, 
available fi:om Omnivision Technologies, hic, 1341 Orleans Drive, Suimyvale, CA, 
USA, providing an output representing an imaged field. 

A user-operated imaging functionality selection switch 2204 is preferably 
provided to enable a user to select operation in a data entry functionality employing the 
electronic imaging sensor module 2202 for data entry responsive to user hand activity. 

The data entry functionality preferably employs a data entry template 
projector 2206, which projects a data entry template, such as that designated by 
reference numeral 2208, onto a surface, such as a table surface 2210. An illuminator 
2212 is operative to illuminate a region adjacent the data entry template. The imaging 
sensor module 2202 views the data entry template region for sensing hght fi-om the 
illuminator 2212 scattered by engagement of a user's finger with the data entry template. 

The data entry functionality also preferably incorporates processing 
functionaUty, an example of which is described in applicant's Published PCT 
Apphcation WO 02/054169 A2, the disclosure of which is hereby incorporated by 
reference. The data entry fimctionality produces a stream of data 2214. 

Reference is now made to Fig. 23, which is a simplified pictorial 
illustration of gaming apparatus constructed and operative in accordance with a 
preferred embodiment of the present invention. As seen in Fig. 23, gaming apparatus, 
one example of which is a board game, such as chess, is provided with an electronic 
imaging sensor module 2302, which may include a CMOS array, for example, CMOS 
array OVT6130, available fi-om Omnivision Technologies, Inc., 1341 Orleans Drive, 
Sunnyvale, CA, USA, providing an output representing an imaged field. 

A user-operated imaging functionality selection switch 2304 is preferably 
provided to enable a user to select operation of one of plural gaming functionaUties, 
such as different board games, employing the electronic imaging sensor module 2302 
for gaming piece locating. 

The gaming functionality preferably employs a gaming template 
projector 2306, which projects a game template, such as a chess board, designated by 
reference numeral 2308, onto a surface, such as a table surface 2310, An illuminator 
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2312 is operative to illuminate a region adjacent the game template 2308. The imaging 
sensor module 2302 views the game template region for sensing hght from the 
illuminator 2312 reflected by optically encoded reflectors 2314 moimted on individual 
gaming pieces 2316. The reflectors 2314 preferably individually identify each gaming 
piece and indicate its location on the game template 2308. 

The gaming functionality also preferably incorporates processing 
functionality, an example of which is described in applicant's Published PCT 
Application WO 02/054169 A2, the disclosure of which is hereby incorporated by 
reference. The gaming functionality produces a stream of data 2316, which may be used 
for game management and scoring. 

Reference is now made to Fig. 24, which is a simplified pictorial 
illustration of gaming apparatus constructed and operative in accordance with another 
preferred embodiment of the present invention. Fig. 24 shows gaming pieces, such as 
dice 2402, which bear variously differently coded reflectors 2404 on various surfaces 
thereof, thereby to enable not only the identity and location but also the spatial 
orientation of the gaming pieces to be sensed. 

It will be appreciated by persons skilled in the art that the present 
invention is not limited to what has been particularly shown and described hereinabove. 
Rather the scope of the present invention includes both combinations and 
subcombinations of the various features described hereinabove as well as modifications 
and variations thereof as would occur to a person of skill in the art upon reading the 
foregoing specification and which are not in the prior art. 
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